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(54) SYSTEM AND METHOD FOR RETRIEVING PICTURE 

(57)Abstract: 

PROBLEM TO BE SOLVED: To improve readability in retrieving and 
displaying plural similar picture candidates from a data base. 
SOLUTION: The clustering of plural pictures is operated by a cluster 
generating part 106 based on the picture characteristics of each picture 
retrieved from a picture characteristics data base 104 by a similar picture 
retrieving part 1 05, and the clustering of the pictures is operated by a 
cluster generating part 106. The plural retrieved pictures are classified and 
displayed for each cluster based on the clustered result by a picture display 
part 107. Therefore, it is possible to display the images belonging to the 
same cluster among the retrieved pictures, that is, the similar pictures 
whose similarity is higher, and to realize display by considering the similarity 
between the retrieved pictures. 



'■Mix 



i 



i 



"V? 



r i 



rrx~: 



:sT; 



LEGAL STATUS 

[Date of request for examination] 23.03.2000 

[Date of sending the examiner's decision of rejection] 

[Kind of final disposal of application other than the 
examiner's decision of rejection or application converted 
registration] 

[Date of final disposal for application] 

[Patent number] 345761 7 

[Date of registration] 01 .08.2003 

[Number of appeal against examiner's decision of 
rejection] 

[Date of requesting appeal against examiner's decision 
of rejection] 

[Date of extinction of right] 



Copyright (C); 1 998,2003 Japan Patent Office 
-1- 



* * NOTICES * 
. JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

- LThis document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] It is the image retrieval system which searches and displays the image stored in the database 
based on a search key. When two or more images are searched according to a search key, based on the 
image description extracted from each image, so that similar images may be contained in the same 
cluster The image retrieval system characterized by providing the cluster classification means which 
carries out the cluster division of said two or more searched images, and a display means to classify and 
display said two or more searched images for every cluster based on the result of the cluster division by 
said cluster classification means. 

[Claim 2] Said display means is an image retrieval system according to claim 1 characterized by having a 
means to adjoin and to display the similar images which were classified according to said cluster 
classification means, and which belong in the same cluster for every cluster. 

[Claim 3] A means to judge whether similarity with the key image which was searched according to said 
search key and which is given as said search key for every image is larger than a predetermined value is 
provided further. Said display means It is the image retrieval system according to claim 1 characterized 
by having the means which arranges in order of the similarity about an image with larger similarity with 
said key image than a predetermined value, and displays, and similarity with said key image classifies and 
displays for every cluster about the image below a predetermined value. 

[Claim 4] It is the image retrieval system according to claim 1 which possesses further a means 
classified according to said cluster classification means to elect a representation image for every cluster, 
and is characterized by said display means having a means to display the representation image 
preferentially for every cluster. 

[Claim 5] It is the image retrieval system which searches and displays the image stored in the database 
based on a search key. A means to determine the images the image description is mutually similar to 
based on the image description extracted from each image when two or more images are searched 
according to a search key, The image retrieval system characterized by providing a display means to 
elect and display a representation image for every similar image group mutually determined that the 
image description will be similar. 

[Claim 6] Said display means is an image retrieval system according to claim 5 characterized by having 
other images, the image which is not similar, and a means to display the representation image elected 
for said every similar image group, in two or more images searched according to said search key. 
[Claim 7] It is the image search method which searches and displays the image stored in the database 
based on a search key. When two or more images are searched according to a search key, based on the 
image description extracted from each image, so that similar images may be contained in the same 
cluster The image search method characterized by providing the cluster classification step which carries 
out the cluster division of said two or more searched images, and the display step which classifies and 
displays said two or more searched images for every cluster based on the result of the cluster division 
by said cluster classification step. 

[Claim 8] It is the image search method which searches and displays the image stored in the database 
based on a search key. The step which determines the images the image description is mutually similar 
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to based on the image description extracted from each image when two or more images are searched 
according to a search key, The image search method characterized by providing the step which elects 
and displays a representation image for every similar image group mutually determined that the image 
description will be similar. 

[Claim 9] It is the record medium with which the computer program for searching and displaying the 
image stored in the database based on a search key was recorded and in which computer reading is 
possible. When two or more images are searched according to a search key, based on the image 
description extracted from each image, said computer program so that similar images may be contained 
in the same cluster The record medium characterized by providing the cluster classification step which 
carries out the cluster division of said two or more searched images, and the display step which 
classifies and displays said two or more searched images for every cluster based on the result of the 
cluster division by said cluster classification step. 

[Claim 10] It is the record medium with which the computer program for searching and displaying the 
image stored in the database based on a search key was recorded and in which computer reading is 
possible. The step as which said computer program determines the images the image description is 
mutually similar to based on the image description extracted from each image when two or more images 
are searched according to a search key, The record medium characterized by providing the step which 
elects and displays a representation image for every similar image group mutually determined that the 
image description will be similar. 

[Translation done.] 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the image retrieval system and image search method 
which have been improved so that the perusal nature of the image searched especially may improve 
about the image retrieval system and image search method which search and display the image stored in 
the database according to a search key. 
[0002] 

[Description of the Prior Art] Development of the image retrieval system for searching and displaying 
the digital image stored in the database in recent years with development of the image processing 
technique by the computer, the spread of digital images, etc. is furthered. An image retrieval system 
searches an image similar to the inputted key image from a database, and shows it to a user. In this 
case, usually a list indication of the searched image is given from an image with high similarity with a key 
image at order. 

[0003] Moreover, the image retrieval system using the narrowing-down retrieval technique of narrowing 
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down gradually the image made into the purpose out of the list display of the searched image is also 

known (JP,10-162020,A). 

[0004] 

[Problem(s) to be Solved by the Invention] by the way, the purpose of image retrieval begins to be used 
[ **** ] also for mainly the same as that of a key image, or the application of extracting many similar 
images from a database as much as possible by including the image whose similarity with a key image is 
not so high recently in the candidate for retrieval although it is said from a database that an image very 
high [ of similarity ] is looked for, comparing, and collating them with a key image . 
[0005] For example, it is the case where it is investigated whether it is the no in which the character 
which it is newly in investigating whether the design of the goods and similar goods are already sold 
when newly developing goods going to use to goods etc., a mark, and a similar thing already exist etc. In 
such an investigation application, since the retrieval of the same image itself is not the purpose, it is 
also required that an image with a partially similar pattern etc. should be looked for. 
[0006] Therefore, it will be necessary to display it also about the image whose similarity with a key 
image is not so high, and to judge a degree, a partial similar part, etc. of similarity with a key image, and 
presentation of many similar image candidates will be required. 

[0007] In image retrieval for such the investigation purpose, sufficient perusal engine performance is not 
expectable by the conventional approach similarity with a key image displays a similar image candidate 
on high order in order. Although indicated by priority about an image with very high similarity with a key 
image, it is because similarity will be judged about a partial similar image which is similar only about a 
certain specific part to be the image which is not so expensive. In this case, since the similarity between 
the images displayed by adjoining each other is also low, the images from which a similar part with a key 
image differs mutually will be located in a line, and they will be displayed, and about such a partial similar 
image, it is actually hard coming to carry out the comparison with a key image. 
[0008] This invention is made in view of an above-mentioned situation, and it enables it to show a 
retrieval result not only in consideration of similarity with a key image but in consideration of the 
similarity of the searched images, and aims at offering the optimal image retrieval system for image 
retrieval and image search methods, such as an investigation application as which presentation of more 
similar image candidates is required. 
[0009] 

[Means for Solving the Problem] In order to solve an above-mentioned technical problem, this invention 
is an image retrieval system which searches and displays the image stored in the database based on a 
search key. When two or more images are searched according to a search key, based on the image 
description extracted from each image, so that similar images may be contained in the same cluster It is 
characterized by providing the cluster classification means which carries out the cluster division of said 
two or more searched images, and a display means to classify and display said two or more searched 
images for every cluster based on the result of the cluster division by said cluster classification means. 
[0010] In this image retrieval system, a cluster part injury line crack of these images and two or more 
searched images are classified and displayed for every cluster based on the image description of two or 
more searched images of each. Therefore, the display which would be possible [ the images i.e., display 
the high similar images of similarity collectively mutually, which belong to the same cluster among the 
searched images ], and took the similarity between the searched images into consideration can be 
performed. 

[001 1] By classifying for every cluster and displaying a retrieval image in the above investigation 
applications, especially, mutually, it becomes possible to perform comparison and collating with a key 
image per cluster unit, i.e., similar images group, such as a pattern, and it becomes possible to aim at 
improvement in investigation effectiveness. 

[0012] Moreover, a means to judge whether similarity of this invention with the key image which was 
searched according to the search key and which is given as said search key for every image is larger 
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than a predetermined value is provided further. Said display means is characterized by having the means 
- which arranges in order of the similarity about an image with larger similarity with said key image than a 
predetermined value, and displays, and similarity with said key image classifies and displays for every 
cluster about the image below a predetermined value. 

[0013] About an image with very high similarity with a key image, the similarity of these images can also 
expect a high thing. It is more effective in improvement in the perusal nature of an image to, observe the 
similarity of the images searched rather than the ranking by the difference of similarity with a key image 
on the other hand, since the similarity of the images searched with the image whose similarity with a key 
image is not so high is not necessarily high. Therefore, similarity with a key image becomes possible 
[ raising the perusal nature of the both sides of a very high image and the image which is not so 
expensive ] by combining a classification display with the display of the order of similarity for every 
cluster. 

[0014] Moreover, this invention possesses further a means classified according to said cluster 
classification means to elect a representation image for every cluster, and said display means is 
characterized by having a means to display the representation image preferentially for every cluster. 
[0015] Thus, it becomes possible by electing and displaying a representation image for every cluster to 
carry out a screen display of more kinds of images at once. Moreover, since it becomes unnecessary to 
peruse other images about the cluster which requires only the comparison with a representation image, 
it also becomes possible to aim at reduction of the number of images which should be perused. 
Furthermore, when the display of other images other than a representation image is required, it becomes 
possible to aim at improvement in perusal nature more by adding the function which adjoins and displays 
other images in the cluster to which the demanded representation image belongs on the corresponding 
representation image. 

[0016] Moreover, this invention is an image retrieval system which searches and displays the image 
stored in the database based on a search key. A means to determine the images the image description 
is mutually similar to based on the image description extracted from each image when two or more 
images are searched according to a search key, It is characterized by providing a display means to elect 
and display a representation image for every similar image group mutually determined that the image 
description will be similar. 

[0017] Thus, when not all the searched images are displayed but images with very high similarity exist in 
a retrieval image, it becomes possible to display only the image with which the image descriptions differ 
mutually as a retrieval result by using the method which displays only the representation image. 
Therefore, the useless efficient image retrieval which is not is realizable. 
[0018] 

[Embodiment of the Invention] Hereafter, the operation gestalt of this invention is explained with 
reference to a drawing. 

(1st operation gestalt) The functional configuration of the image browsing system concerning the 1st 
operation gestalt of this invention is shown in drawing 1 . This image browsing system is for perusing 
searching a digital image from a database, and searches and carries out a screen display of the image 
similar to the key image inputted as a search key from a database. 

[0019] This image browsing system discovers a key image, an image similar on the whole, an image with 
a partially similar part, etc. out of the image about the trademark registered [ for example, ], and is used 
for the application of comparing and collating it with a key image. In order to offer the optimal perusal 
engine performance for such an application, the function to show the images which were similar to the . 
image browsing system of this operation gestalt not only in consideration of similarity with a key image 
but the similarity of the searched images on the occasion of the display of a retrieval image collectively 
is prepared. Hereafter, the concrete functional configuration of an image browsing system is explained. 
[0020] The image browsing system of this operation gestalt consists of functional modules, such as the 
image input section 101, the image feature-extraction section 102, the image description storage 
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section 103, the image description database 104, the similar image retrieval section 105, the cluster 
generation section 106, and the image display section 107, like illustration. 

[0021] The image input section 101 is a module used for the input of the image which should be 
registered into the image description database 104, and the key image used as a search key, for example, 
has the function to incorporate the pattern on space as image data, or to incorporate the image file 
created by computer using a scanner. 

[0022] The image feature-extraction section 102 extracts the image description from each image data 
inputted through the image input section 101. Here, the image description (feature extracted from 
image) shows characteristic quantity which each image has, such as a shape facility and the color 
description. It can use for arbitration combining the parameter with various amount of the white and the 
black pixel in the predetermined division field in the whole image or an image or its rate, numbers of runs, 
rates of an aspect ratio of outside **, numbers of connection fields, etc. as characteristic quantity of an 
image. About the image which should be registered into the image description database 104, the image 
description extracted by the image feature-extraction section 102 is recorded on the image description 
database 104 with the inputted subject-copy image data. On the other hand, about the key image used 
as a search key, the image description extracted by the image feature-extraction section 102 is sent to 
the similar image retrieval section 105. 

[0023] It is carried out by comparing and collating the image description of each image stored in the 
image description in which it is and this retrieval was extracted from the key image and the image 
description database 104 for the similar image retrieval section 105 to search an image similar to a key 
image from the image description database 104. Each image searched by the similar image retrieval 
section 105 from the image description database 104 is sent to the cluster generation section 106. 
[0024] The cluster generation section 106 is for carrying out the cluster division of these images based 
on the image description of each of all searched images, and it classifies each searched image into two 
or more clusters so that images with the similar image description may be contained in the same cluster. 
A cluster is the group who packed similar images and images of each other with the near description 
distance are classified into the same cluster on an image feature space. 

[0025] Based on the cluster classification result of the retrieval image obtained by the cluster 
generation section 106, the image display section 107 classifies each retrieval image for every cluster, 
and displays it on a monitor. In this case, in order to aim at improvement in the perusal nature of the 
image by the user, the high similar images of similarity are displayed collectively, the images, i.e., each, 
who belongs to the same cluster among the searched images. 

[0026] the flow chart of 2 — referring to — **** of construction processing of the image description 
database 104 — it ****** just. First, the image for registration is inputted in the image input section 
101 (step ST 101), and the input image is sent to the image feature-extraction section 102. In the image 
feature-extraction section 102, the image description is extracted from an input image (step ST 102), 
and additional record of the image description is carried out by the image description storage section 
103 at the image description database 104 (step ST 103). (Step ST104) and the image description 
database 104 are built by doing this activity repeatedly to all input images. 

[0027] With reference to <similar image retrieval and a display process>, next the flow chart of drawing 
3 , the procedure for similar image retrieval and a display is described. First, the image specified as a 
key image k is inputted by the user in the image input section 101 (step ST 201), and the key image k is 
sent to the image feature-extraction section 102. In the image feature-extraction section 102, the 
image description is extracted from the key image k (step ST 202). The image description extracted 
from this key image k is sent to the similar image retrieval section 105, and the processing which 
searches a similar image from the image description database 104 is started (step ST 203). 
[0028] here — the similarity of the key image k and the registered image i in the image description 
database 104 — S (k, i) — with however (S (k, i)>=0), if it carries out, the image which is S(k, i) >T will 
be searched by the similar image retrieval section 105 as a similar image candidate out of the image 
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description database. T is taken as a similarity threshold here. What is necessary is to just be referred 
to as T= 0 in case all images are browsed. 

[0029] Then, a cluster division is performed so that images with the similar image description may be 
contained in the same cluster based on distribution of the image characteristic quantity about each 
similar image candidate searched with the cluster generation section 106 (step ST 204). As the 
technique of cluster generation, distribution between clusters is large and the technique of setting up a 
cluster so that cluster internal variance may become small, the technique of carrying out grouping of the 
image with the near description distance one by one, etc. can be considered, for example. The example 
of the image cluster in which the cluster division was carried out by the cluster generation section 106 
is shown in drawing 4 . 

[0030] The example at the time of using the two-dimensional image feature space of X and Y is shown 
by drawing 4 . The number of dimension of an image feature space supports the number of the 
parameters used as an index of the characteristic quantity of an image, and may use the image feature 
space beyond a three dimension in fact. Clusters A, B, and C are image clusters in which the cluster 
division was carried out by the cluster generation section 106. Moreover, a, b, c, d, e, f, g, and h are 
similar image candidates who fill S(k, i) >T, and are taken as S(a, k) >S(b, k) >S(c, k) >S(d, k) >S(e, k) 
>S(f, k) >S(g, k) >S (h, k). Thus, the searched similar image candidate will be classified according to the 
image description, i.e., a category. 

[0031] Then, in the image display section 107, a similar image candidate is collectively displayed for 
every cluster using the result by which the cluster division was carried out (step ST 205). In this 
category display, the approach which the center-of-gravity location of the image description of the 
image which constitutes a cluster displays sequentially from the image description of the key image k 
and a near cluster should just be used for the order of a display of a cluster. This is realizable by 
performing processing (step ST 302) whose center-ol^gravity location of the image which reads first the 
image which belongs there for every cluster one by one from the image description database 104 (step 
ST 301), then constitutes a cluster displays an image side by side sequentially from the cluster near the 
key image k as shown in the flow chart of drawing 6 . 

[0032] Supposing the center of gravity of a cluster is close to the image description of the key image k 
in order of Clusters A, B, and C in the example shown in drawing 4 , a retrieval image will be displayed 
for every cluster like drawing 5 (A) in order of (a, c, g), (b, d, h), and (e, f). The images b, d, and h which 
the images a, c, and g contained about Cluster A there have the mutually similar description, and are 
contained about Cluster B there have the description which was mutually similar about another point 
which is different in Images a, c, and g. The same is said of Cluster C. 

[0033] As mentioned above, since images with the image description which was similar with this 
operation gestalt adjoin compared with the conventional technique which displays an image in order of 
similarity with the key image k simply as shown in drawing 5 (B) by showing similar images collectively 
not only in consideration of similarity with the key image k but in consideration of the similarity of the 
searched images and it is displayed, it has the advantage from which perusal of an image becomes easy. 
[0034] That is, a user can observe two or more images with a common partial similar point with the key 
image k collectively, and can perform easily comparison and verification with the key image k by viewing. 
Therefore, the especially optimal image browsing for investigation of a trademark etc. becomes possible. 
Of course, it can use effectively also for the investigation purposes, such as a microorganism and a 
virus, by accumulating images, such as not only investigation of a trademark but microphotography, in 
the database. 

[0035] (The 2nd operation gestalt) Next, the 2nd operation gestalt of this invention is explained with 
reference to drawing 7 thru/or drawing 9 . In case the image browsing system of a **** 2 operation 
gestalt presents the key image k and a similar image, it is characterized by changing the priority of the 
order display of similarity, or a category display according to the value of similarity with the key image k, 
and is the same as that of the 1 st operation gestalt about construction of the image description 
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database, similar image retrieval, and cluster generation processing. 

[0036] That is, with the 2nd operation gestalt, in case image display is performed in the image display 
section 107, priority is given to "the order display of similarity" about an image with very high similarity 
with the key image k, and is given to "a category display" about an image with similarity low to some 
extent. It explains concretely using drawing 7 . 

[0037] Like drawing 7 , even when the cluster generation section 106 separates into the cluster of A, B, 
and C, about an image with very high (S(k, i) > T2) similarity with the key image k, priority is given to the 
display of the order of similarity like a or b, and is given to a category display about the remaining image 
(c, d, e, f, g, h) with similarity low to some extent. Here, T2 shall fill the relation of T2>T. 
[0038] The example of a display by this operation gestalt is shown in drawing 8 . Like illustration, it is 
excepted from the display group for every cluster, is displayed independently about the images a and b 
with very high similarity with the key image k, and is displayed in the condition of having classified for 
every cluster, about the other images c, d, e, f, g, and h. 

[0039] It is expectable that Images a and b of the similarity between these images a and b are also 
expensive since similarity with the key image k is very high. Therefore, also in the display gestalt of 
drawing 8 , a mutually similar image will be located in a line, and will be displayed. Moreover, it can be 
considered that it is an image similar on the whole to the key image k, and Images a and b are images 
similar as partially about c, d, e, f, g, and h as the key image k. Therefore, it becomes possible by 
combining the display of the order of similarity, and the category display for every cluster to raise the 
perusal nature of the both sides of an image similar on the whole to the key image k, and an image 
similar to a partial target. 

[0040] Next, with reference to the flow chart of drawing 9 , the procedure of the image display 
processing in a **** 2 operation gestalt is explained. 

[0041] First, in the image display section 107, it is judged about each image searched by the similar 
image retrieval section 105 whether it is the image which fills S(k, i) >T2 (step ST 31 1). The similar 
image retrieval section 105 can perform this judgment based on the similarity computed for every image 
at the time of retrieval. S(k, i) > It will be classified into "a category display" about the image which is 
classified into "the order display of similarity" about the image which fills T2 (step ST 312), and does 
not fill S(k, i) >T2 (step ST 313). 

[0042] Then, in the image display section 107, each searched image is displayed according to the above- 
mentioned display classification ("the order display of similarity", "category display") determined for 
every images of these (step ST 314). 

[0043] Since the images which the similarity with the key image k could give priority to the high image 
with the **** 2 operation gestalt by the above, and could display, and were mutually similar by the 
image with comparatively low similarity adjoin, it has the advantage from which perusal of an image 
becomes easy. 

[0044] (3rd operation gestalt) Next, the 3rd operation gestalt of this invention is explained with 
reference to drawing 10 thru/or drawing 15 R> 5. When two or more retrieval of the very similar image is 
carried out, the image browsing system of a **** 3 operation gestalt displays only those representation 
images, is characterized by omitting the display of the remaining image, and is the same as that of the 
1st operation gestalt about construction of the image description database, similar image retrieval, and 
cluster generation processing. 

[0045] That is, in the image registered into the image description database 104, two or more registration 
of the same image or the image which does not have an almost difference in a pattern may completely 
be carried out. Then, when the image with the almost same image description exists, it becomes possible 
by displaying only the representation image to be able to reduce the number of the images to peruse 
and to display more kinds of images on a screen. It explains concretely using drawing 10 . 
[0046] In drawing 10 , the case where the image characteristic quantity of Image a and Image b is very 
similar among the images searched by the similar image retrieval section 105, and those image 
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characteristic quantity is very similar also about Image d, Image e, and Image f is shown, such — what - 
- one image is elected as a case as a representation image for each [ these ] similar image group of 
every, only the representation image is displayed on it, and processing which omits the display of other 
remaining images is performed. The example of a display in this case is shown in drawing 1 1 R> 1. 
[0047] In drawing 1 1 , the case where d is displayed as a representation image is shown about the 
similar image group who a is displayed as a representation image about the similar image group who 
consists of a and b, and consists of d, e, and f. Moreover, it is displayed independently about the image c 
without a very similar image. In order to show that there are two or more very similar similar images, you 
may trim around a representation image. 

[0048] Approaches, like as the selection approach of a representation image, similarity with the key 
image k chooses the highest image as a representation image from the image groups belonging to the 
same similar image group can be used. Of course, since the similarity with the key image k hardly 
changes between the images belonging to the same similar image group in many cases, you may make it 
choose one image as a representation image at random. 

[0049] Drawing 12 is the 2nd example of a display. Here, as shown in drawing 10 , although Image c is 
classified into Cluster B and the pan as a cluster C and, as for Images a and b, Cluster A and Images d, 
e, and h are very similar to them about Cluster A, Images a, b, and g Although g of the similarity with a 
and b is [ Images d, e, and f ] very similar about Cluster B similarly so highly, h assumes the case so 
that the similarity with d, e, and f may not be so high. 

[0050] In this case, although the category display according to Clusters A and B and C is performed like 
the 1st operation gestalt, about Cluster A, all the images a, b, and g are not displayed, the 
representation image a elected from Image g and the images a and b which constitute a similar image 
group is displayed, and b is not displayed. Moreover, the representation image d which are not all the 
images d, e, f, and h, and was elected also about Cluster B from the images d, e, and f which constitute 
Image h and a similar image group is displayed. 

[0051] In addition, when the cluster generation section 106 sets up a cluster with very small cluster 
internal variance, all the images that belong in the same cluster become mutually very similar things. 
Therefore, what is necessary is to elect only one image as a representation image in such a case, and to 
display it on it for every cluster. Thus, the example of the procedure for electing and displaying a 
representation image for every cluster is shown in drawing 13 . 

[0052] First, the image display section 107 determines an image with the highest similarity with the key 
image k in the image which was generated by the cluster generation section 106 and which belongs to it 
for every cluster, and elects it as a representation image (step ST 321). For example, when a cluster 
division like drawing 10 is made, if attached to Cluster A, about Cluster C, Image c will be elected 
[ Image a / Cluster / B ] for Image d as a representation image. 

[0053] Then, the image display section 107 displays the representation image elected for every cluster 
side by side (step ST 322). Thereby, a retrieval image is expressed as the same display gestalt as 
drawing 1 1 . 

[0054] Next, the procedure in the case of browsing the image with which the display is omitted from a 
representation image with reference to the flow chart of drawing 14 is explained. 

[0055] If a specific representation image is chosen by click actuation of the mouse by the user etc. on 
drawing 1 1 or the display screen of drawing 1 2 (YES of a step ST 331), the image display section 107 
will read other images in the cluster to which the selected representation image belongs from the image 
description database 104 (step ST 332), and it will carry out a screen display of the read image so that 
the selected representation image may be adjoined (step ST 333). The example of a display in this case 
is shown in drawing 15 . 

[0056] Drawing 15 is an example when the representation image d of Cluster B is chosen. In this case, in 
other images belonging to Cluster B, the remaining images e and f (or images e, f, and h) which were not 
elected as the representation image d and a very similar image, i.e., a representation image, are read 
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from the image description database 104, and are displayed. 

[0057] As mentioned above, with a **** 3 operation gestalt, when two or more mutually very similar 
images exist, by displaying only a representation image and omitting the display of the remaining similar 
image, the number of the images to peruse can be reduced and it has the advantage which can display 
more kinds of images on a screen. 

[0058] As mentioned above, although the image browsing system feature of this invention was divided 
into the 1st thru/or the 3rd operation gestalt and was explained, the system having all the display 
functions of these [ 1st ] thru/or each 3rd operation gestalt may be realized, and a display gestalt may 
be chosen if needed. For example, the display gestalt of the 1st operation gestalt is usually used, and 
when there are very many searched images (i.e., when there are very many images per one cluster), 
improvement in perusal nature can be realized more by switching to the display gestalt of the 3rd 
operation gestalt etc. 

[0059] Moreover, each image browsing system of the 1st thru/or the 3rd operation gestalt is realizable 
as software which can be performed on a computer. In this case, it becomes possible to acquire easily 
the same effectiveness as this operation gestalt only by introducing a computer program including that 
procedure into the usual computer through the record medium in which computer reading is possible. 
[0060] 

[Effect of the Invention] As explained above, according to this invention, it has the following 
effectiveness. 

[0061] 1) Since similar images adjoin compared with the conventional technique which displays an image 
in order of similarity with a key image simply, perusal of an image becomes easy. 

[0062] 2) Since the images which the similarity with a key image could give priority to and display the 
high image, and were mutually similar by the image with comparatively low similarity adjoin, perusal of an 
image becomes easy. 

[0063] 3) When two or more very similar images exist, by displaying only a representation image and 
omitting the display of the remaining similar image, the number of the images to peruse can be reduced 
or more kinds of images can be displayed on a screen. 



[Translation done.] 



* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] The block diagram showing the functional configuration of the image browsing system 
concerning 1 operation gestalt of this invention. 

[Drawing 2] The flow chart which shows the procedure of construction processing of the image 
description database in the image browsing system of this operation gestalt. 

[Drawing 3] The flow chart which shows the procedure of the similar image retrieval and the display 
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process in the image browsing system of this operation gestalt. 

[Drawing 4] Drawing showing an example as a result of the cluster division performed by the image 
browsing system of this operation gestalt. 

[Drawing 5] Drawing showing an example of the retrieval result display screen in the image browsing 
system of this operation gestalt. 

[Drawing 6] The flow chart which shows an example of the retrieval result display process procedure in 
the image browsing system of this operation gestalt. 

[Drawing 7] Drawing showing another example as a result of the cluster division performed by the image 
browsing system of this operation gestalt. 

[Drawing 8] Drawing showing other examples of the retrieval result display screen in the image browsing 
system of this operation gestalt. 

[Drawing 9] The flow chart which shows other examples of the retrieval result display process 
procedure in the image browsing system of this operation gestalt. 

[Drawing 10] Drawing showing still more nearly another example as a result of the cluster division 
performed by the image browsing system of this operation gestalt. 

[Drawing 1 1] Drawing showing the example of further others of the retrieval result display screen in the 
image browsing system of this operation gestalt. 

[Drawing 12] Drawing showing another still more nearly another example of the retrieval result display 
screen in the image browsing system of this operation gestalt. 

[Drawing 13] The flow chart which shows the example of further others of the retrieval result display 
process procedure in the image browsing system of this operation gestalt. 
[Drawing 14] The flow chart which shows the procedure of the browsing processing from the 
representation image in this operation gestalt. 

[Drawing 15] Drawing showing another still more nearly another example of the retrieval result display 
screen in the image browsing system of this operation gestalt. 
[Description of Notations] 

101 — Image input section 

102 — Image feature-extraction section 

103 — Image description storage section 

104 — The image description database 

105 — Similar image retrieval section 

106 — Cluster generation section 

107 — Image display section 



[Translation done.] 
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&^;i/-yKJSf •5B&i¥©f B«k £©Si!{H 

^frjRfSBfitMfcfetiTtt, *-B<gik £:©Sti£(«te 

ai^'^b^&u^a^u©-?, a&ff&ic— o©b 

[0 0 4 9] H12ttS2®*g«T»5. iCTtt, 
01 OlC^-fi^lC B#a, b, g^7^^A, B 

£UTa*anT*S9. ^7^^AlCOV»TttB«a, 
bW«l:MlTt^*?- gtea, bt©S«lif 
nS£iR<XC<, HttHcd^X^BfcoUTfeBBd. 
e, f M&ttCSMHLTt>«#. httd, e, f t<Dm 

[0 0 5 0] d©*§£\ £57.* A, b, csij©*^^ 

Xi'ACODTIt B{£a, b, gOiTi*8fan* 

ma, boD«p*»&aaianfcf^*B*a©*-««»*a 

Bftd, e, f, hCD4TTitt<, iiht, SiKB 
mif)l-7$:ffil$.-?%W&d, e, f ©if^&afflSft 
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// 

[0 0 5 1] &i5, 9yZ?£.tfLffll 0 6tf?7X?fa 
h<Dm±ttLZ. <fcoT. Z\<D£ott®'&\Zl,${7?7>9 

mz i o©H«o**«*iiii t vxmth UTSsvrn 

[0 0 5 2] £1*, H&g^gBl 0 7«. ^7,37 

ST 3 2 1) „ 01 OOi 

**, ^X^BlCO^TteB&da*, ^^X^CtCOti 

rimmed ftsiiitLTiuisnsuitft*. 

[0 0 5 3] l&^T, flfl&^ffil 0 7tt. #^^7^ 

3 2 2). z\ti\z&<o, m 1 1 tmcMifi&mT'&mm 

[0 0 5 4] 0 1 4 ©7 a — ^-V— h^tlL 

T, *^*^Bg;*nT^3B^£ttSB{£7^:7*^X 

[0 0 5 5] 01 1 S&ttHl 2©^tkBB±T\ 3.— 

«*»a*?3nst (Xfy7'ST3 3 lOYES) , B 

o 7«> aj?snfcft*ii«i**«-f <e>^7^ 

Uf7^ST3 3 2) , ^©f!#.>AA/£niH§!£, 

t77"st3 3 3) . z.ftm&o^m&m l 5 

To 

[0056] inn ^5^^B0Dftaa«fed*«a*? 

ffi©H#<Dft»T, tt»B«d£#fiffc:«MHLfcB«'3i: 
Dft«B»£UTafflSftfc!0>-3fc»D©B«e, f 
(fcSWiBttie, f, h) dtIM»f-i"<-Xl 

0 4^e>^*-2i*nT^$ns. 

[0 0 5 7] £U:©<fc3fc:. *I3*»ITH S«r» 

LfcB**M*»#fcT*«£\ 

0. BIK-r*iB«i©»*B'J«T?#, £<9#<©fflgi©B 
«*BBK**TS**iJ££#:3. 
[0 0 5 8] £U:. *I^©It^7fy>^->XfA 

* sfc 0 ©B*»3i«##»c#^«^fctt, MS 3 §H 



(7) 

P*3Ktt©fa±£llglT£3. • 
[0 0 5 9] Sfc, B175S*3£16»iB©B«:77? 
v 5 > i/->X t- A tt <t*n t> =3 > bf a - 9 ±T5SfT"J«fc V 

© ¥ )M * £ tr a > tT a - * 7 s n ^ 5 A * rj > tT n - * tR 
*Bl 0 pJflg&gBfltKtt&a CTilST© 3 > fcf a. (C© 

A-fsrwc, ^mmmtmm<D9j}^^\zm^z 

[0 0 6 0] 

T©3&S£WT5. 
[0 0 6 1 ] 1) !M*C*-B«i©S«lSEJIiK:B«S: 

[0 0 6 2] 2) *-B«£©S<H«#H^Btfi£«5te 

[0 0 6 3] 3) ##fc«{HLfcB«#*»#fc-r*« 
20 ft«B«!©#-£S^U »B©S{HB«©B*£* 
KT*::i:K:J:D. n«T*B«©*£H"J«T€rfc?K 
«fcO#K©«B©B«£B®K:BjR-?#a. 

[0ffi©B¥fcifcBJ§] 

[0 1 ] *mi<D—&M»WXz.%*wm.'f?VV>>fi' 
[02] H*MSJ^»OB«^5'>>?>^^7 : Afc*Jtt 

•sBttwax-^^-^o^ssfflao^JBs^r^n- 

[0 3] IDHSS^^WH^^^^^^^vX^At'fett 
30 fiRMHBfttfHR • a*»lffl¥iftST7D-fir- 
K 

[04] nHlt»ttOB*^?9i?>^<>^7AiCJ:-3 

[0 5] |B|*JS^^©B#77'7v>^i/7.xAtCi5lt 
ft«HRIi*«*BiBO— «ft«TH. 

[0 6] |SlllSSP^©B^7'7'7> J >^v7>5 i A^i5lt 
5tifeS*SHfli®-«s|^St7n-ft- K 

[0 7] |W|^ffi^©B«y7'7v>yvX^A^j:o 

io [0 8] mmmmm<Dmmyyo^>if^7,f L A\z^-f 
[09] mnmmm t <Dwm77'yi?>?i'XT&\z&rt 
k 

[010] ra^Jfi»SB©B«^7'>>>>^->^7 i i»lC«fc 
o Tfrfctlfc £ 5 X ^#tt©iiSS©£ e Kjw©B*3Sf 
0. 

[011] mnmmiR<Dn»7'7'>z?>ifi'XTJ*x.t$ 
so [0i 2] mnmBm<Dwm77'5i;>tf>'7sf-2±\z& 
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(8) 



13 

[0i3] m j MMMm<Dmm-?7<5i>>tfi/*TA\z& 
[015] mmmmm'Dmmzf^'y'jyifiy^L.izio 
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(9) 



[0 5] 



[17] 



(A) 




S0O)>T 2 <OttH 




[0 9] 



[0 10] 



ST311 
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[0 13] 



ST321 



* ? 7 * * » & « jsftt* <n & n mift * 
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(10) 



[HI 5] 




(51) Int. CI. 7 ttS<J8B*l FI f-73-K (##) 

G 0 6 T 7/00 3 0 0 G 0 6 T 7/00 3 0 0 F 

F^-A(##) 5B050 EA18 FA02 FA12 FA19 GA08 
5B075 ND06 NK02.NR12 PP22 PQ02 

PQ46 PQ48 PQ74 PR06 QM08 
5L096 BA08 DA04 FA60 JA03 JA22 

MAO 7 



